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157] ABSTRACT 

Multiple media push engines communicate with the multi- 
media client through a mulli casting network that may 
incorporate multiple delivery paths. Hie streaming data 
representing media selections tor delivery are distributed 
across multiple media push engines using a non-hicrarehial 
coding technique in which the data are represented as a set 
of sub-stream components, capable of being reconstituted 
from fewer than all of the components of the original data 
stream. The higher the number of components used in 
rcconstitution, the higher the quality of service is provided 
by the reconstituted stream. Admission control to the group 
multicast session is administered in a distributed fashion, 
where an admission control unit opens the multicast stream, 
with all subsequent admission control decisions being made 
by the media push engines themselves. Substream compo- 
nent data are sent using Real-lime transport protocol while 
session management and the distributed admission control 
process arc administered under the Real-Time Control Pro- 
tocol. 

15 Claims, 9 Drawing Sheets 




02/20/2003, EAST Version: 1.03.0002 



U.S. Patent jui.27, 1999 sheet 1 of 9 5,928,331 




02/20/2003, EAST version: 1.03.0002 




02/20/2003, EAST Version: 1.03.0002 



U.S. Patent jui.27, 1999 sheet 3 of 9 5,928,331 



Input 
X 




Xi X2 x n 

_TH (ZiZ 2 ) = (1 + aZi) (1 + aZ 2 ) 
L.where o < a < 1 

FIGURE 3 



02/20/2003, EAST version: 1.03.0002 



U.S. Patent 



Jul. 27, 1999 



Sheet 4 of 9 



5,928,331 



a. 
a 
3 



CL 

a. 

CL 



Q. 
Q. 

Q. 



a) 
E 

0) 
-C 

UJ 



0) 

E 

UJ 









ATM 


CL 


TCP 


Q. 




t 




Ethernet 



a. 








t 


CL 








o 




















a> 






CL 


E 
a> 
















LU 


a 


CL 








D 








3 







fO 



■Ti- 
ro 



ro 



■o 

ro 



02/20/2003, EAST Version: 1.03.0002 



U.S. Patent jui.27,1999 sheet 5 of 9 5,928,331 



RTP Packet Format 



8 



16 



31 



Payload Type 



Sequence Number 



Time Stamp 



Synchronization Source ID# (SSRC) 



Contributing Source ID# List (CSRC) 



RT Application - Specific Data 



FIGURE 5 



02/20/2003, EAST version: 1.03.0002 



U.S. Patent j u i. 27, 1999 



Sheet 6 of 9 



5,928,331 




02/20/2003, EAST Version: 1.03.0002 




02/20/2003, EAST Version: 1.03.0002 




02/20/2003, EAST Version: 1.03.0002 



U.S. Patent 



Jul. 27, 1999 



Sheet 9 of 9 



5,928,331 




(0 <D 



55 



A 



CO 
=* 

CO 
CO 



CL 



CV] 

St: 

O 
< 



E 

o 

C 

CO 
CD 



A 



w 

o 



CM 

CL 
H 



CL -5 c UJ 
UJ 



02/20/2003, EAST version: 1.03.0002 



5,9: 

l 

DISTRIBUTED INTERNET PROTOCOL- 
BASED REAL-TIME MULTIMEDIA 
STREAMING ARCHITECTURE 

BACKGROUND AND SUMMARY OF THE 
INVENTION 

The present invention relates generally to networked 
multimedia systems. More particularly, the invention relates 
to a media delivery system for delivering media selections to 
one or more media clients over a multicasting network. 

With the explosive growth of the Internet, there is a 
growing interest in using the Internet and other Internet 
protocol-based networks to deliver multimedia selections, 
such as video and audio material. Interactive television, 
movies on demand, and other multimedia push technologies 
arc among the more promising applications. 

T*hc Internet is a connectionless network offering best 
effort delivery service. Packets of data are routed as data- 
grams thai carry the address of the intended recipient. A 
specific connection between the sender and the recipient is 
not required, because all host nodes on the network include 
the inherent capability to route datagrams from node to node 
until delivery is effected. This datagram packet delivery 
scheme is constructed as a best effort delivery system in 
which the delivery of datagram packets is not guaranteed. 
Datagram packets may be sent via different routes in the 
effort to increase the likelihood of delivery. Thus, if one 
node on the network is experiencing congestion, subsequent 
datagrams may be alternately routed to avoid the congested 
node. This means that data datagram packets do not inher- 
ently have a guaranteed arrival time. Even packets corre- 
sponding to a single message may be received out of order. 
This fact significantly affects how certain multimedia data 
are delivered. 

In many cases, multimedia data require real-time delivery. 
In the case of audio or video data, the data stream repre- 
senting a particular media selection needs to be delivered in 
the proper time sequence, to allow the user to play back the 
audio or video selection "live" as it is being sent. Clearly, if 
the datagram packets are delivered out of order, due to 
taking different delivery routes, then playback at the multi- 
media client (e.g., a user's interactive TV) will be jumbled. 

The Real-time Protocol (RTP) is a current dc facto stan- 
dard for delivering real-time content over the Internet (or 
other networks based on an IP protocol). The Real- lime 
Protocol replaces the conventional transmission control pro- 
tocol (TCP) with a framework that real-time applications 
can use directly for data transport. Currently, the RTP 
standard supports a first type of message, namely one for 
carrying the media content data or streaming data. Typically, 
a separate protocol, the Real-Time Control Protocol (RTCP) 
is used with RTP to pass control messages for session 
management, rale adaptation and the like. 

While the Real-time Protocol can be used to deliver 
multimedia streaming data over computer networks, the 
existing architecture has not proven robust enough to pro- 
vide high quality presentation using best effort network 
services such as those provided by the Internet. The present 
invention solves this problem by using a distributed media 
push architecture thai is capable of supplying streaming tlala 
redundantly from multiple sources and over multiple distri- 
bution paths. The media push engines have associated media 
storage units that store streaming data as non-hierarchial sets 
of substream components. The components are capable of 
being reconstituted into a reconstructed stream from fewer 
than all of the components, such lhat the higher the number 
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of components used in reconslilution, the higher the quality 
of the reconstructed stream. 

Conventional systems use a hierarchial coding scheme 
that treats some components as more important than others. 

5 Thus, conventional systems typically need to expend con- 
siderable resources to guarantee that the more important 
components are always delivered. In contrast, the media 
delivery system of the invention uses a non-hierarchial 
coding scheme, multiple description coding (MDC) that 

10 treats all components as equals. Thus, no special resources 
need to be allocated to ensure that a given set of substream 
components is delivered. Naturally, the higher number of 
components delivered, the higher the quality achieved; on 
the other hand, unlike with conventional hierarchial coding, 

15 loss of any given single packet does not appreciably degrade 
the signal quality. 

The distributed media delivery system also employs a 
distributed admission control system. The media client con- 
tacts a single admission control unit to request a given media 

20 selection, but thereafter the admission control decisions are 
handled in distributed fashion by the media push engines 
themselves. The admission control unit communicates the 
request to a plurality of media push engines distributed 
across the network and those push engines individually 

25 determine whether they can participate in the multicasting 
session. Thus, the individual media push engines each 
evaluate local traffic congestion to determine whether it is 
capable of supplying the requested data stream. The admis- 
sion control unit is thus not involved in directly determining 
which media push engines should be admitted to a multicast 
group session. The admission control unit simply assigns the 
multicast group session address and then allows the admis- 
sion process to proceed autonomously, in a distributed 
fashion. 

35 

I- or a more complete understanding of the invention, its 
objects and advantages, reference may be had to the follow- 
ing specification and to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

40 

FIG. 1 is a network datagram illustrating a preferred 
embodiment of the invention; 

FIG. 2 is a detailed network datagram showing how two 
different data streams (X and Y) arc distributed across the 
45 network using non-hierarchial multiple description coding; 

FIG. 3 is a data How diagram illustrating one embodiment 
of multiple descriptive coding (MDC); 

FIG. 4 is a layer datagram illustrating the TCP/IP archi- 
tecture and also showing how the RTP architecture may be 
50 integrated into an IP-based system; 

FIG. 5 is a detailed format datagram illustrating the packet 
formal according to the Real-lime Protocol (RTP); 

FIG. 6 is a network datagram illustrating the call admis- 
^ sion and session management according to the invention; 

FIG. 7 is a detailed network datagram showing informa- 
tion How between a media push engine and a multimedia 
client over ihe multicasting IP network; 

FIG. 8 is a network datagram illustrating the process of 
tiu source component server redistribution; and 

FIG. 9 is a protocol datagram showing how the Real-lime 
Protocol (RTP) may be modified to increase its reliability. 

DETAILED DESCRIPTION OF THE 
65 PREFERRED EMBODIMENT 

Referring to FIG. I, an exemplary distributed networked 
multimedia system is illustrated at 10. A plurality of media 
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push engines 12 are accessible through the multicasting 
network 14. The presently preferred embodiment is 
designed to work over a network employing the Internet 
protocol (IP); however, the principles of the invention may 
be readily extended to networks using other protocols. The 5 
network 14 is also assessable by one or more multimedia 
clients 16, as illustrated. An admission control unit 18, 
accessible through network 14, performs certain admission 
control functions, primarily to initiate or open a multicast 
group session. The admission control unit includes a catalog m 
services system 20. The catalog services system contains a 
database record indicating which multimedia selections are 
available for delivery by the plurality of media push engines. 
Although involved in opening a multicast group session, the 
admission control process is actually performed in a distrib- 15 
uted fashion as will be more fully discussed below. 

The distributed media delivery system responds to deliv- 
ery requests from a multimedia client by opening a multicast 
group session among the media client and those multimedia 
push engines that have the requested media selection avail- -jq 
able for delivery. Typically multiple media push engines will 
participate in simultaneously delivering streaming data cor- 
responding to the requested selection. The multimedia client 
is a user host that performs the presentation function. It 
reconstructs a final stream from the various stream compo- 25 
nents delivered by the participating media push engines. 
Each media push engine has its own data storage for the 
stream component data and those data storage systems can 
be mediated by a suitable distributed file system that pro- 
vides a mou nlable and transparent storage and retrieval ^ 
function. 

An important aspect of the distributed media delivery 
system is the manner in which streaming data are stored on 
the media push engines. Unlike traditional systems that store 
multimedia data in a hierarchial fashion, the present inven- 35 
tion uses a non -hierarchial coding scheme, referred to here 
as multiple description coding (MDC). The multiple 
description coding splits the video and/or audio stream into 
substrcams called components. Each component can then be 
coded and transmitted over the network independently from 40 
all other components. The client software on a multimedia 
client 16 can assemble a reconstructed stream from any 
subset of the components. Thus the reconstructed stream can 
be assembled from fewer than all of the components. The 
higher the number of components used in reconstruction, the 45 
higher the quality of the reconstructed stream. 

Using this non-hierarchial coding to deliver streaming 
data over the inherently unreliable network affords surpris- 
ingly robust media delivery, particularly when multiple push 
engines participate in the delivery. As will be more fully 50 
discussed, the media push engines control the multicast 
group session admission process themselves, in a distributed 
fashion, adding or subtracting media push engines to the 
group session as needed to maintain a high quality of 
services. Thus, when the multicasting network 14 exhibits 55 
low traffic congestion, only a few media push engines may 
be needed to supply all of the components of the MDC- 
encoded stream. Even if some components are not delivered 
in a timely fashion, the multimedia client will nevertheless 
be able to reconstruct the stream for presentation (with some 60 
degree of degraded quality). IlThe network traffic congestion 
is high, the media push engines negotiate with one another 
to add additional media push engines. Because the admis- 
sion control process is distributed, individual media push 
engines are able to assess their own local traffic congestion 65 
and will thus participate in the group session, or not, 
depending on local traffic conditions. 
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FIG. 2 shows in greater detail how the multiple descrip- 
tion coding works. In FIG. 2, two multimedia streams, 
designated X and Y are stored across a plurality of media 
push engines. These streams are broken into substream 
components, designated by subscripts, X r , X 2 , . . . , X„; Y,, 
Y 2 , . . . Y /( . Note that the substream components stored 
across the plurality of media push engines arc not necessar- 
ily the same for each engine. Thus media push engine 12a 
stores components X,, X ft and Y,. Similarly, media push 
engine 12b stores components X 2 , X 7 and Y 2 . 

The multimedia clients reassemble the data stream of 
interest by summing the proper substream components in the 
proper order. Thus multimedia client 16a reconstructs 
stream X as illustrated, while multimedia 16b constructs 
stream Y as illustrated. At the multimedia client it matters 
not that individual substream components arrive through 
different paths from different media push engines. 

The presently preferred multiple description coding 
scheme is constructed as illustrated in FIG. 3. The original 
multimedia data stream (e.g. video and/or audio data) is 
decomposed into multiple subsignals and then each subsig- 
nal is independently compressed. As noted above, the 
decomposition is non-hierarchial, such that an acceptable 
signal can be recovered from any one subsignal, an incre- 
mental improvement can be realized by additional 
subsignals, and a perfect reconstruction is achieved when all 
subsignals are received exactly. Moreover, it is preferable to 
maximize the overall compression gain so long as the above 
three criteria are met. 

One way to decompose the original signal is to construct 
each subsignal as a reduced resolution representation of the 
original signal. This may be done, as illustrated, by submit- 
ting the original signal through a low pass filler followed by 
downsampling. The subsignals thus differ only in the sam- 
pling positions. If desired, such decomposition can be 
obtained by a filter bank that includes the pre-filter and its 
shifted versions. 

The pre-filter should be selected to suppress the aliasing 
components in the clow nsam pled subsignals. This helps to 
reduce the bit rates required for coding the subsignals and to 
enable an acceptable image recovery from a single subsig- 
nal. 

The pre -filter should further be selected so that it does not 
completely eliminate the high frequency components. Were 
the high frequency components entirely eliminated, there 
would be no way to recover those components in the original 
signal, even when all subsignals are present. 'Hi us the filler 
should suppress, but not entire I y eliminate, the high fre- 
quency components. 

Mathematically, reconstruction of the substream compo- 
nents into a reconstructed stream involves inverting the 
matrix equation relating the samples in all the subsireams 
and the samples in the original stream. Typically this may 
involve a large matrix equation, with inversion employing a 
large amount of computation and memory space. One way 
to address this computational burden is to use a block 
recursive reconstruction method. At each step in the recur- 
sive process a block of 2x2 samples in the original stream is 
recovered based on up to four corresponding samples in the 
received subcomponent streams. Of course, other computa- 
tional techniques may be employed to accomplish the same 
result. For more information on coding and encoding in a 
non-hierarchial fashion using multiple description coding, 
see "Robust Image Coding and Transport in Wireless Net- 
works Using a Non-Hierarchial Decomposition," Yao Wang 
and Doo-man Chung. Mobile Multimedia Communications; 
Goodman (Plenum Press) 
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The M DC-coded subslreams are delivered over the mul- supplied from entily 30 lo entily 38 through the network 

ticasting network as datagrams using the Real-Time Proto- consisting of entities 30, 32, 34, 36 and 38. 

col (RTP) for content delivery and the Real-Time Protocol xi ie RTP p rol0 col is designed for multicast operation. 

(RTCP) to implement How control. These protocols are able Multicasting is a form of message broadcasting in which 

to naturally coexist with the popular TCP/IP protocol used 5 messages may be delivered to many different recipients in a 

by many Internet applications. FIG. 4 presents a review of designated set. Multicast addresses identify sets of 

these protocols through an example of how five entities interfaces, frequently including multiple interfaces belong- 

might communicate with one another. In FIG. 5 entities in& t0 different systems. When a message has a multicast 

30-38 communicate with each oilier using the layered destination address, the network strives to deliver it to all 

architecture popularized by the Internet. It will, of course, be m interfaces in the set. This function lets a system generate a 

understood thai FIG. 4 is intended only to show how the message once and have that message delivered to many 

presently preferred Real-Time Protocol (RTP) integrates in different recipients. 

one possible architectural scheme. Allhough the Real-Time Asjde fmm de | jveri lhe dal kels l0 mul|i , e 

Protocol .s presently preferred, this .s no. intended as a recipienISi a mu i lic asling network will typically also support 

limitation ot .he invention ,n its broader aspects. On the is feedback from , he m recipients. Typically all partici- 

contrary, other message delivery protocols may be suitably pams jn , hc mullicasling group XS!iion can rcccivc lhcsc 

11 " feedback messages. Such feedback messages are commonly 
In FIG. 4 each of the communicating entities 30-38 has used f or real-lime traffic control, following a related Real- 
been illustrated as a layered architecture, with the physical 7j me Control Protocol (RTCP). In some respects, RTCP is 
layer at the bottom and the application layer at the top. Entity 20 an optional extension to RTP RTCP packets are used by the 
30 communicates with entity 32 using the Ethernet Protocol presently preferred embodiment to send flow control' and 
at the physical level. Entity 32 and entity 34 communicate session management information among the entities partici- 
at the physical level using the ATM Protocol. In a like pat j ng m , ne group multicast session, 
fashion, entity 34 communicates with entity 36 using the F , G 5 i]lustrales the RTP packel formaL Nole that the 
Ethernet Protocol, and entity 36 communicates with entity 25 e( indudes a sequence mimber ant , (ime sl usu| in 
38 using the PPP Protocol. Again, the physical layer com- reassembIilH . lhe packels in the proper time sequence. 

munication protocols selected for illustration here are not " • * ■ .i i t i_ i . ■ r 

...... . • • . • . FIGS. 6 and 7 show the details of how the multimedia 

intended as a limitation upon the invention asset forth in the .. , . . . , , . , .. . 

, , . . 1 client, admission control unit and media push engines corn- 
appended claims. . , i * . . . l to 

™ municatc with one another during a multicast group session. 

Above the physical layer is the Internet Protocol (IP) - Specifically, FIG. 6 shows the basic message flow and 

layer. The IP Protocol isolates the physical or transport layer communication sequence of the preferred embodiment. FIG. 

from the application layer. The IP Protocol supports con- 7 givus a morc dclailud vicw of how lnc RTp and RTCp 

nectionless communication m which packets ot information Protocols are used in routing the substream component 

are sent and received as datagrams. Note that in lhe illus- ^ dalagrarns from media push engine to multimedia client. 

traled example ot FIG. 4 all communicating entities are " n . r rir , ^ lU , . .. t . t£ , 

1 & Referring first to FIG. 6, the multimedia client 16 sends a 

using the 11 1 rotocol. unicasl Tcp Pro|ocoI mcssagc t0 thc Admission Control 

One layer above the IP Protocol two different higher level Unjt 18) requcslin g mat a particular media selection corn- 
protocols have been illustrated, the TCP Protocol and the mence delivery. The Admission Control Unit 18 consults its 
UDP Protocol. Again, the illustrations are intended only as 4f) cata log services system 20 to determine if the requested 
one example of a possible configuration. Thc UDP Protocol selection (i.e. requested stream) is present on the network, 
or User Datagram Protocol represents a simple transport Assuming the stream is present, the Admission Control Unit 
protocol. It makes no attempt lo preserve the sequence of transmits a Stream Open message lo those Media Push 
messages it delivers. The TCP Protocol or Transmission Engines 12 that have at least some substream components of 
Control Protocol provides a higher level of reliability, yet 4 , thc rcqucstcc ] selection. This Open Stream request is sent to 
insures that datagrams are delivered in the proper sequence. all Media Push Engines. Those Media Push Engines that find 
The TCP Protocol uses an acknowledgment system to insure themselves capable of serving the requested stream compo- 
that all datagrams arc delivered in the proper sequence. Thc nents. jointly enter (he multicasting session management and 
TCP Protocol includes a mechanism for retransmitting pack- flow conlro ] seS sion between only those hosts serving and 
els that have not been acknowledged. This acknowledgment/ ^ rece j v ing that specific stream. Participating Media >ush 
retransmit technique guarantees proper packel delivery, but Engines and participaling multimedia clients obtain the 
it does not guarantee packet delivery in real-time. Thus TCP nccdcd mu i ticasl addrcss for such control multicast uroup 
Protocol is usually unsuitable for delivering real-lime data session from lhe Adm j ss i on Control Unit. Thereafter, the 
such as multimedia video and/or audio data. Admission Control Unit effectively drops out of the session, 

The Real-Time Protocol (RTP) replaces the more soph is- 55 allowing subsequent session management and flow control 

licated Transport Protocol of TCP with a simple framework messages to be exchanged only among multicast group 

that applications can use directly. Instead of implementing a members (that is, the multimedia client and all correspontl- 

missing data detection and retransmission mechanism — ing media push engines). This reduces the overhead on the 

which can introduce transmission delays — the RTP Protocol Admission Control Unit IS. 

simply ignores missing data. The RTP Protocol is also not 6u The Admission Control Unit generates a multicast Class 

typically concerned with the sequence of packet delivery. rj address r or „ se by the multicast session. This address may 

The protocol assumes that the application layer above it will fc e selected from a pool of available multicast address 

correct any misordered data. The RTP Protocol is compatible entries. Hie Admission Control Unit is thus responsible for 

with^ a number of different encoding standards, such as managing the allocation of multicast addresses. When a 

MPEG, JPEG and H. 261. 65 multicast session is ended the Admission Control Unit 

In the illustrated example of FIG. 4, entities 30 and 38 arc returns thc multicast session address back to the pool of 

both running RTP Protocol. Thus streaming data could be available addresses. 
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Thus once the multicast group session is initialed, those 
media push engines able to supply substream components 
will do so by sending unicast RTP session stream data to the 
multimedia client 16. Through the RTCP Protocol these 
media push engines may communicate with one another to 5 
join or depart a multicast group session, as needed to 
maintain a high quality of service. 

Referring to FIG. 7, the media push engine and multime- 
dia client communicate through the network at two different 
levels. As illustrated by the dotted lines, a unicast RTP in 
session transmits the multimedia streaming data to the 
multimedia client. Concurrently, as required, the media push 
engine and multimedia client send each other RTCP reports, 
specifically sender reports and receiver reports as well as 
any appropriate How control commands and other session 15 
management commands (e.g. Start Push, Pause, Continue). 
The RTCP control signals are shown by the bidirectional 
solid line in FIG. 7. 

Essentially, after the Stream Open message is sent by the 
Admission Control Unit, each mediapush engine consults its 20 
associated media storage system to determine whether it is 
capable of serving the requested stream components. If so, 
then the media push engine joins the specified multicast 
group. Otherwise it docs not participate in the multicast 
session (unless later requested). Once the media push engine 2> 
has joined the multicast group, it participates in communi- 
cations using the RTCP Protocol whereby statistics of sent 
data and received data are exchanged among the members of 
the group. As noted above, the Admission Control Unit does 
not need to participate in these communications and hence 30 
it will remain dormant unless a request for another session 
is made or until the current session is requested to be 
terminated. 

Effectively, the system implements a distributed Admis- ^ 
sion Control System, where ihe participating members of the 
group collectively and distributive^ make the admission 
control decisions. One benefit of this distributed approach is 
that the invention can incorporate intelligent mechanisms to 
prevent network congestion and to improve quality of 4Q 
service, despite the fact that the multicasting network is a 
best effort network with no guaranteed real-time delivery. 

Best effort networks, particularly those lacking sophisti- 
cated traffic and user control policies, experience frequent 
congestions. Such congestions can result in a loss of or 45 
substantial delay of real-time data. As previously discussed, 
real-time data that is delivered late is effectively treated as 
not delivered. The continuous influx of data into a congested 
node of the network tends to make the congestion even 
worse. The present invention uses RTCP sender reports and 50 
received reports on time-sensitive transmissions as an indi- 
cation that a given node may need to scale back on the 
number of components it is transmitting when congestion is 
detected. 

FIG. 4 illustrates how this may lie accomplished. Hie 55 
multimedia client 16 has requested the X real-time data 
stream, consisting of sub-stream components Xj, X 2 , X 3 and 
Y 4 . Assume that Media Push Engine 12 in FIG. 7 is 
experiencing local traffic congestion such that sub-stream 
components are arriving late at the multimedia client 16. I*he 60 
multimedia client's RTCP receiver report notifies the Media 
Push Engine 12 (and all other media push engines partici- 
pating in the group session) that some percentage of the 
component data from Media Push Engine 12. Media Push 
Engine 12 analyzes these reports and stops sending a 65 
selected component, in this case the X 3 , thereby decreasing 
the amount of traffic flowing through its point of congestion. 
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Thus, after adjustment, Media Push Engine 12 supplies only 
components X lf X 2 and X 4 to the multimedia client. As the 
other media push engines participating in the group session 
receive the same sender and receiver reports, the loss of the 
X 3 component from Media Push Engine 12 may be com- 
pensated for if other push engines are able to supply this 
missing component. Otherwise, the quality of service will be 
slightly degraded as discussed above. 

FIG. 8 illustrates how the data stream may be effectively 
redistributed by making local adjustments to the substream 
components being sent. In the illustrated example assume 
that there is local congestion somewhere in the data path that 
supplies substream components from Media Push Engine 
12/?. The RTCP sender and receiver reports will thus indicate 
that some portion of the components previously sent to the 
multimedia client 16 by Media Push Engine 126 arc lost or 
delayed due to local congestion. In the illustrated example 
the lost components happen also to be present in the storage 
system of Media Push Engine 12a. Media Push Engine 12a 
can either retransmit the lost component payloads to the 
multimedia client or simply adjust the set of components to 
be transmitted in future real-time data transactions. In the 
case where lost payloads arc retransmitted by another media 
push engine, sufficient buffering should be supplied at the 
multimedia client to allow the missing components to be 
reassembled with the previously delivered components 
before the stream is reconstructed and presented to the user. 
In the case where the system merely alters the component set 
for future transmissions, such change constitutes a scalable 
server component redistribution mechanism. This mecha- 
nism promotes improved quality of service by improving the 
presentation of stream data to the multimedia client. 

Although the embodiment described above is generally 
suitable for most media delivery applications, there are some 
systems that cannot tolerate even a slight degradation in 
quality. Such systems include high quality broadcast video 
distribution. In these more demanding applications, the 
system of the previously described preferred embodiment 
can be modified to employ an additional reliability mecha- 
nism for the real-time component. In this case the Real-Time 
Protocol may be modified or augmented to allow retrans- 
missions of lost real -time payloads. This "Reliable RTP" is 
illustrated in FIG. 9. The Media Push Engine communicates 
to the RTP slack using Real-Time Protocol. In this case 
assume that the first and third components are received but 
the second component is missing. There is an immediate 
negative acknowledgment (NACK) from the RTP stack, 
telling the Media Push Engine that the second payload was 
not received. The Media Push Engine then retransmits the 
needed payload and the RTP stack places the needed payload 
into the correct location within the data buffer. The client 
application then reads the data from the dala buffer. Any 
duplicate packets are dropped and any excessively delayed 
packets may also be dropped. 

From the foregoing it will be understood that the inven- 
tion provides a media delivery system architecture thai 
employs a distributed, networked technique for delivering 
streaming dala over a best effort network. The architecture 
may be readily scaled, larger or smaller, as the server's 
complexity increases linearly with the number of clients. 
The architecture is thus a fully distributed, tightly coupled 
parallel architecture that is capable of providing simple and 
yet robust service. 

Through the use of multiple description coding (MDC) 
and multiple path transport, the invention can provide a high 
quality of service without resorting to delay-producing 
transmission retry techniques. Thus ihc invention will 
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readily work with existing Real-Time Transport Protocol 
(RTP) for data transport and Real-Time Control Protocol 
(RTCP) for session management, rale adaptation and the 
like. When congestion is encountered the presentation can 
be downscaled without interruption, thanks to the multiple 5 
description coding and the manner in which participants of 
a group session can be added to or removed from the group. 
The How control of streams is also controllable through 
these same mechanisms to prevent or reduce network con- 
gestion once it is detected. in 

The invention is therefore ideally suited for delivery of 
multimedia selections as well as video and audio streaming 
data. The invention will readily support data streams of 
multiple bit rates and it is capable of providing services at 
both constant bit rates and variable bit rates. 15 

While the invention has been described in its presently 
preferred embodiments, it will be understood that the inven- 
tion is capable of certain modification and change without 
departing from the spirit of the invention as set forth in the 
appended claims. 

What is claimed is: 

1. A distributed media delivery system for delivering 
media selections to a media client over a multicasting 
network, comprising: ^_ 

a plurality of media push engines accessible through said 
network, said push engines each having associated 
media storage unit for storing streaming data represent- 
ing the media selections available for delivery; 

said media storage units being con figured to store said $q 
streaming data as a non-hierarchical set of substream 
components capable of being reconstituted into a 
reconstructed stream from fewer than all of said 
components, such that the higher the number of com- 
ponents used in rcconstitution, the higher quality the 35 
reconstructed stream; and 

an admission control system accessible through said 
network, said admission control system including a 
catalog for storing the identity of the media selections 
available for delivery by each of said media push 40 
engines, 

said admission control system being operative, in 
response to a request for a given media selection from 
a media client, to open a multicast group session among 
said media client and at least a portion of said media 45 
push engines having the given media selection avail- 
able for delivery, 

whereby said media push engines participating in said 
multicast group session each supply to said network 
those substream components corresponding to the 
given media selection, for delivery to and reconstitu- 
tion by said media client. 

2. The media delivery system of claim 1 wherein said 
admission control system includes an admission control unit 
that maintains a pool of multicast session addresses for use 
in invoking multicast group sessions and wherein said 
admission control unit assigns a designated multicast ses- 
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sion address, selected from said pool, for use by said 
multicast group session. 

3. The media delivery system of claim 2 wherein said 
admission control unit is further operative, in response to a 
request from said media client to end a multicast group 
session, to return said designated multicast session address 
to said pool. 

4. The media delivery system of claim 2 wherein said 
media client and said media push engines participate in said 
multicast group session exchange flow control messages 
without involving said admission control unit. 

5. The media delivery system of claim 1 wherein at least 
a portion of said substream components are replicated across 
first and second media push engines and wherein said 
delivery system further comprises congestion handling sys- 
tem for identifying one of said first and second media push 
engines as the cause of congestion and for automatically 
invoking the other of said first and second media push 
engines to participate in said multicast group session. 

6. The media delivery system of claim 5 further compris- 
ing data buffering system associated with said media client 
for storing substream components prior to reconstitution. 

7. The media delivery system of claim 1 wherein said 
admission control system is a distributed system defined at 
least in part through interaction between said media push 
engines. 

8. The media delivery system of claim I wherein said 
media push engines communicate with said network over 
different communication paths. 

9. The media delivery system of claim 1 wherein said 
network is a connectionless network offering best effort 
delivery services. 

10. The media delivery system of claim 1 wherein said 
network is the Internet. 

11. The media delivery system of claim 1 wherein said 
media client and said media push engines participating in 
said multicast group session employ Real-Time Transport 
Protocol (RTP) for data transport. 

12. The media delivery system of claim 1 wherein said 
media client and said media push engines participating in 
said multicast group session employ Real-Time Control 
Protocol (RTCP) for session management. 

13. The media delivery system of claim 1 wherein at least 
a portion of said substream components are replicated across 
several media push engines. 

14. The media delivery system of claim I wherein said 
admission control unit is further operative, in response 10 a 
request from said media client to end a multicast group 
session, to instruct all media push engines participating in 
said multicast group session to terminate the session. 

15. The media delivery system of claim 1 wherein 
between said media client and every media push engine 
participating in said multicast group session there is a 
unicast How of datagrams containing real-time stream com- 
ponent data. 
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